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1. Introduction 

We propose a new framework of shift dynamics, called functional shifts, which extends 
symbolic dynamics. A functional shift is defined as a shift space that is a set of bi- 
infinite sequences of some functions on a set of symbols, while symbolic dynamics is 
usually defined as a set of bi-infinite sequences of finite symbols. This functional shift 
generates another shift space, called a generated shift, as follows. Consider a sequence of 
functions (/i)iez contained in the functional shift. The sequence of functions generates 
a sequence of symbols (xj)jgz determined by Xj+i = fi{xi). A set of such bi-infinite 
sequences of symbols is also a shift space. Thus, this framework gives us a method to 
analyze the relationship among classes of shift spaces using the generative operation. 

Introducing the framework of functional shifts allows us to consider the dynamic 
change of functions. In traditional dynamical systems theory, the time dependence of 
a function governing change of states is not serious. The reason for is that time in 
a dynamical system can be treated as an additional phase space variable, so that for 
any dynamical system we can describe it with a time independent function. However, 
the time dependency of functions, i.e., the dynamic change of functions, has recently 
become a subject of interest in the investigation of dynamical systems, because there 
are many phenomena in which functions should be regarded as variable. For example, 
in a population dynamics with species extinction and speciation, the degree of freedom 
dynamically changes [H In studying such phenomenon, since it is difficult to get 
an immutable evolution rule, we want to treat functions of the system as dynamic. 
Therefore, considering dynamic changes of function is an important perspective for 
understanding complex systems. We often call such dynamic changes meta-dynamics. 

Several models have been proposed to study the dynamic change of functions 
in 13 El El El El • Sato and Ikegami (Sj introduced switching map systems, in which 
maps to govern the evolution of the systems are dynamically switched with other maps in 
the system. Kataoka and Kaneko (110101 investigated the evolution of a one- dimensional 
function /„ defined by fn+i = {l — e)fn + efnofn- Studying dynamics in which functions 
vary in time using meta-dynamics can be important when considering systems evolution 
or learning. Fontana [7] studied abstract chemical systems that are defined by a loop in 
which objects encode the functions that act on them. For another instance, Tsuda [H] 
proposed a switching map system as a model of the brain. He has shown that a skew- 
product transformation can be considered as a framework describing meta-dynamics. 
Functional shifts also can be represented by skew-product transformations. 

The framework of functional shift has two major advantages. One is the ability 
to directly compare dynamics with meta-dynamics, since both are represented by 
shift spaces. The other is to be able to analyze both dynamical and computational 
characteristics, because this framework is an extension of symbolic dynamics. Moreover, 
as a kind of meta-dynamics, we can discuss self-modifying systems in terms of functional 
shifts, in which the functions governing the dynamics of the system are used to change 
the functions themselves. 
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In this paper, we study functional shifts from the viewpoints of both dynamics 
and computation. In recent years, several studies have focused on the relationship 
between dynamics and computation [HIl HH [121 UHl HH UHl HEl UZj- The central idea in 
these studies is to regard the time evolution of dynamics as a computational process. 
Based on a correspondence between the unpredictability of dynamical systems and 
the halting problem, Moore JU] insists on the existence of dynamics that are more 
complex than chaos. In addition, computational complexity for continuous time analog 
computation has been studied ^21 ^1 which the convergence of ordinary 

differential equations is interpreted as a process of computation. The relationship 
between the complexity of dynamics and computation is, however, still unclear. In 
this work we discuss the complexity of the dynamics of functional shifts in terms of 
topological entropy, which measures the diversity of orbits of dynamical systems, and 
also investigate the complexity of their computation in terms of the Chomsky hierarchy. 
Through this study, we argue that dynamics and computation give us opposite results 
concerning the complexity of systems. 

In an analysis of the complexity of computation in functional shifts, we prove that 
there exists a shift space whose language is not recursively enumerable (r.e.), even though 
the language of the functional shift that generates it is r.e. Computational classes of 
sets to be beyond r.e. are related to analog computation in the interest of both dynamics 
and computation. While Siegelmann ^Tj introduced analog shifts as a model of analog 
computation which is more powerful than the universal Turing machine, the dynamical 
features of such a powerful computation system are an open problem. When we study 
dynamical systems modified by meta-dynamics, we discuss analog computation with 
functional shifts, and argue that the existence of dynamical behaviour that is more 
complex than the universal Turing machine should be taken into account. 

This paper is organized as follows. We first review some basic definitions of shift 
spaces, and define functional shifts in section |21 Next, in section IHl we investigate the 
property of entropy in functional shifts. We show that the entropy of a functional shift 
gives the upper limit for that of a generated shift given by it, in order to study the 
relationship between dynamics and meta-dynamics (see theorem 13. ip . In section |3J we 
compare functional shifts with generated shifts, by focusing on how the language of a 
shift space belongs to the Chomsky hierarchy of formal languages. We prove that any 
class of the languages of functional shifts is contained in that of the generated shifts 
given by them (see corollary 14.21 and theorem 14. 4| 14. 7| and I4.l2|) . One of the most 
important results in this section is that there is a shift space whose language is not 
r.e., even though the language of a functional shift to generate the shift space is r.e. (see 
theorem I4.12|) . Finally, our results are discussed from the standpoints of dynamics, 
computation, meta-dynamics, and self-modifying systems. 
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2. Definition 

Since we will study shift dynamics, we first give some definitions for shift spaces [iT^ . 
Let ^ be a nonempty finite set of symbols called an alphabet. The full ^-shift (simply 
the full shift) is the collection of all bi-infinite sequences of symbols from A. Here such 
a sequence is denoted by x = (xj)igz and the full ^-shift is denoted by 

A^ = {x = (x,),ez|V« eZx.e A}. (1) 

A block over ^ is a finite sequence of symbols from A. An n-block is simply a block 
of length n. We write blocks without separating their symbols by commas or other 
punctuation, so that a typical block over A = {a, b} looks like aababbabbb. A sequence 
of no symbols is called an empty block and denoted by e. For any alphabet A, we write 
A* to denote the set of all blocks over A. If x G A^ and i < j, then we denote a block 
of coordinates in x from position i to position j by X[ij] = XjXj+i ■ ■ - Xj. 

Let 0" be a shift map on a full shift A^ defined by cr((aj)jgz) = (ai+i)jgz for any 
a G A^. A subset X of A^ is called shift-invariant iff cr(X) = X. Let F, which we call 
the forbidden blocks, be a collection of blocks over A. For any such F, define X^ to be 
the subset of sequences in A^ in which no block in F occurs. A shift space is a subset X 
of A^ such that X = Xp for some collection F. Note that if X is a shift space then X is 
shift-invariant, but a shift-invariant set is not necessarily a shift space (some examples 
are in ReffTSj). The set of all n- blocks that occur in points in X is denoted by Bn{X), 
and the language of X is the collection B{X) = [J'^=oBn{X). The language of a shift 
space determines the shift space. Thus two shift spaces are equal iff they have the same 
language. 

Suppose that X is a shift space and A is an alphabet. An {m + n + l)-block map 
$ : Bm+n+i(yX) A maps from allowed (m + n + l)-blocks in X to symbols in A. A 
map (j) : X ^ A^ defined hj y = 0(x) with yi = ^{xi_rnXi-m+i ■ ■ -Xi+n) is called a 
sliding block code induced by $. If F is a shift space over A and 0(X) C Y, then we 
write : X — > y. If a sliding block code (p : X ^ Y is onto, (f) is called a factor code. 
A shift space F is a factor of X if there is a factor code from X onto Y. A sliding block 
code : X ^ y is a conjugacy from X to y if it is bijective. Two shift spaces X and 
Y are conjugate (written X = F) if there is a conjugacy between X and Y. 

Next, we define functional shifts and generated shifts, and explain the basic property 
of each. 

Definition 2.1 Let ^ be a nonempty finite set, and F be a set of maps on A. A 
functional shift JF is a shift space which is a subset of the full shift F^. 
A generated shift Xjc- given by is defined by 

X:f = {{Xi)i^^ G A^\ 3(/,)iez G .F G Z x^+i = fi{xi)}. (2) 

Although a generated shift is not required by our definition to be a shift space, it 
is always a shift space. 

Theorem 2.2 If JF is a functional shift, Xjr is a shift space. 
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Proof. Let 

Y^ = {xe ^"1 V/ G Bn{T) 3t ^ f\{x,)} (3) 

and F = |J„gpjF„. Suppose that X is a shift space which can be described by the 
collection F of forbidden blocks. If x G Xjc, then x G X because every block in F does 
not occur in x. Thus Xjr c X. 

Conversely if x G X, then X[-n,n] ^ F for all n because F is a set of blocks never 
occurring in points in X. Therefore, 

Vn G N 3/ G -B2„(J^) Vi X[_„,„]j+i = fi{xi_n,n]i), (4) 

then 

3f e Wi e Z Xi+i = fi{xi), (5) 

so that X G Xjr. Accordingly X C Xjr. Hence X = Xjr and Xjr is a shift space. i 

By theorem 12. 2^ a functional shift is regarded as a rule to generate a shift space. 
Since any functional shift is also a shift space, we can compare functional shifts with 
generated shifts by using the properties of shift spaces. 

The following examples are instances of functional shifts which generate shift spaces. 



Example 2.3 (Full shifts) Let A = {0, 1} and F = {/, g} be a set of maps such that 



X 


fix) 


g{x) 





1 





1 





1 



If a functional shift is equal to F^, then Xjr is the full shift A^. 

Example 2.4 (Golden mean shift) Let X be the set of all binary sequences with no 
two I's next to each other, so that X = X^, where F = {11}. This is called the golden 
mean shift. 

Let A = {0, 1} and F = {/, ^f} be a set of maps such that 



X 


fix) 


g{x) 





1 





1 









If a functional shift is equal to F , then Xyr is the set of all binary sequences not to 
contain contiguous I's. Thus Xjr is equal to the golden mean shift X. 

Example 2.5 (Sturmian shifts) Consider the circle map 

T(x) = X + a mod 1 (6) 
with irrational a G [0, 1]. Let S C {0, 1}^ be a set 

S = {se {0,lf |xG [0,l),Vn G Z s„ = [T"(x)/aJ}, (7) 

where [xj is the integer part of x. S is not necessarily closed, but it is shift-invariant, 
and so its closure X^ = C\{S) is a shift space, called a Sturmian shift. 
Let F = {/, (7} be a set of maps given by 
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X 


fix) 


g{x) 





1 


1 - [2a\ 


1 





1 - [2a\ 



and Xa is a Sturmian shift with irrational a. Here JF 
$ : {0, 1}2 ^ F is defined by 



f if X = 01 or X 
g otherwise, 



10, 



(f){Xa), where a 2-block map 



(8) 



and a map : Xa is a shding block code induced by $. Since a is irrational, 

00 or 11 must appear in x, so that for any h E there is a unique sequence x G Xq, 
such that = /i and = hn{xn) for all n E'L. Therefore this functional shift T 
satisfies = Xa and Xjr = Xa- 



3. Entropy 

This section will describe the properties of the relationship between functional shifts 
and generated shifts by analyzing the entropy for those shifts. 
The entropy of a shift space X is defined by 

h{X) = lim - \og^\Bn{X)\. (9) 

Note that logg \A\ is the upper limit of h{X) for any X C J^. The entropy of X 
is a measure of the growth rate of the number of n-blocks occurring in points in X. 
Furthermore, if a distance function (i of X is determined by 

] 2-\^\ ii Xk^ Vk Xi = Viioi -\k\ <i <\kl 

^^"'^) = |o ifx = ,, 

then the entropy of X is equal to the topological entropy of the shift map on the metric 
space (X, d) . Hence we regard the entropy of a shift space as the topological entropy. 

It is known that the topological entropy is an indicator of the complexity of the 
dynamics and that it is invariant under topological conjugacy. The existence of positive 
topological entropy implies that a system is chaotic, because the topological entropy 
measures the mixing rate of the global orbit structure of the system. 

The following theorem is an important result of the entropy of functional shifts. 

Theorem 3.1 If JF is a functional shift, then h(Xjr) < /i(jF). 

Proof. Let '■ Bn{J^) 2-^" be defined by 

iPnif) = {xe Bn{X:p)\ 3aeAxo = fo{a) A Wi x^+i = fi+i{xi)}, (11) 

where 2^ denotes the power set of X. It is clear that \^nif)\ < \A\ for all / G Bn{J^) 
and 

i3„(X^)c U v^„(/). (12) 
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Thus |i3„(X^)| < \B„{J^)\\A\. Accordingly, 
/i(X^)= lim -log2|i3„(X^)| 

n^oo n 

< lim -\og,i\BniT)\\A\) 
= lim -log2 \Bn{J^)\ 

n— »oo n 

= h{T). (13) 
Hence h{Xr) <h{J^). , 

Example 3.2 Let be a functional shift given in example 12 .41 Here hlJ-") = log2 2 
and h{X^) = log^ (this derivation formula is in Ref PI). Thus h{X^) < h{J^). 

The key point in the proof of theorem 13.11 is to satisfy a condition |;B„(Xjf)| < 
|;B„(J-')||^| for any G N. The condition implies that for any sequence of symbols 
X G Xjr there exists one or more sequences of functions / G which are restricted by 
Xn+i = fn{xn) for all n G Z. Since the plural sequences (in some case, it is infinitely) in 
JF can correspond to a sequence in Xjf, the entropy of JF is the same as or larger than 
that of Xjr. 

If F is a set of maps on A such that 

^f,geF f y^g^\fxeA f{x) + g{x), (14) 

then every functional shift T C satisfies /i(Xjr) = /i(JF), because |i3„(Xjr)| > \En{T)\ 
for all n G N. Hence 

3/, G F 3x G ^ / ^ A /(x) = g{x:) (15) 

is a necessary condition in order to realize h{Xjr) < h{J^). For example, if |^| < |F| 
and JF = F^, then F satisfies equation (fTH|) and h[Xjr) < h[J^). However, there exists 
a case in which F satisfies equation (fT^ and h{Xjr) = h{J^). The example 12.51 is one 
of such instances, because JF and Xjf are conjugate and the entropy is invariant under 
conjugacy. 

From theorem 13.11 we may consider that the degree of complexity of a functional 
shift JF is greater than that of Xjr from the viewpoint of dynamics. However, satisfying 
such a relationship is not necessarily required in the computational point of view. 
The next section turns to the computational power of shift spaces, and compares the 
languages of functional shifts with those of generated shifts. 

4. Computation in functional shifts 

In sectional we compared the complexity of functional shifts with generated shifts based 
on entropy. Entropy measures the exponential growth rate of the number of orbits 
distinguished in limited accuracy, i.e., it represents sensitivity to the initial conditions 
of a dynamical system. On the other hand, there is the complexity of languages given 
by the Chomsky hierarchy which differs from that of entropy. The complexity described 



Dynamics and computation in functional shifts 



8 



by the Chomsky hierarchy is based on the memory size of the automata that recognize 



regular languages which do not need any memory; context free languages which have 
just a stack; context sensitive languages which have a storage capacity proportional to 
the input word length; and type-0 languages which have unlimited memory. 

The memory size of an automaton is deeply related to the long-range correlation and 
unpredictability of a dynamical system. Badii and Politi have discussed the relationship 
between memory size and the properties of dynamical systems with some examples of 
physical systems corresponding to formal languages in the Chomsky hierarchy ^H] • For 
example, random walks with two reflecting barriers are dynamics whose languages are 
regular, because the domain surrounded by two barriers can be considered to express 
a finite state. Those dynamics can be described by the Markov graph, so they are 
stationary and ergodic. Random walks with one reflecting barrier are dynamics whose 
languages are context free, because the domain on the semi-lattice that makes one 
barrier the starting point can be considered to express a stack. The fact that there is no 
restriction about distance from a barrier brings long-range correlation to those random 
walks. For another example, self-avoiding random walks are dynamics whose languages 
are more complex than context free languages. Some dynamics whose languages are 
not context free have strong unpredictability. In section we discuss this property in 
detail. 

In this section, we compare functional shifts with generated shifts, by focusing on 
how the language of a shift space belongs to the Chomsky hierarchy of formal languages. 
Notice that not every collection of blocks is the language of a shift space. Namely, if X 
is a shift space and w G B{X), then 

• every subblock of w belongs to B{X), and 

• there are nonempty blocks u and v in B{X) such that uwv G B{X). 

Given a class of languages of functional shifts a class of languages of generated 
shifts ^ is given by ^ = {B{Xjr)\ G is a functional shift}. Now we consider 

the inclusion relation between ^ and 5f . From the properties of entropy in functional 
shifts, we may consider that ^ is at least as complex as ^ from the dynamical viewpoint, 
because for any shift space X whose language is in Q there exists a functional shift !F 
whose language is in ^ satisfying Xjr = X, so that h{X) < h{J-'). Thus, if the 
complexity of entropy corresponds to that of computation, we shall expect W as the 
subclass of It is, however, known that the complexities of entropy and computation 
generally do not correspond. For instance, consider the language of a periodic shift 
space which only contains periodic sequences and that of a full shift. Both are contained 
in a class of regular languages which is the lowest computation class in the Chomsky 
hierarchy. However, the former has minimal entropy, and the latter has maximal entropy. 
The fact that shift spaces with different entropy belong to the same computational class 
makes it generally difficult to clarify the relationship between entropy and the complexity 
of computation. Hence it is worthwhile to investigate this relationship using functional 
shifts. We will show the inclusion relation between ^ and in the case where ^ 




In this sense languages are classified into four classes: 
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and ^ belong to the Chomsky hierarchy, and discuss the complexity of dynamics and 
computation with these results. 

We first prove the following theorem to be a basic principle of functional shifts. 

Theorem 4.1 For any shift space X over A, there is a functional shift over F and 
a 1-block map $ : ^ — > F such that 

• Xr = X, 

• $ is a one-to-one mapping, 

• induced by $ satisfies (t){X) = T . 

Proof. For any a G ^, we define a function fa'.A^Ahy 

yxeA fa{x) = a. (16) 
Let F = {fa\a G A}, and $ be defined by 

Va G ^ $(a) = fa. (17) 

Clearly $ is a one-to-one mapping, so that (j){X) is a shift space. Now a functional shift 
J-' is defined by JF = (j){X). Then 

X e X:f ^3f e J^Wi eZ x,+i = fi{x,) 

^3feJ^yteZ^x,+i) = fi 

^xex. (18) 

Thus X^ = X. , 

From theorem 14.11 we can get the next corollary. 

Corollary 4.2 Let ^ be a class of languages of shift spaces. Suppose that ?f is a class 
of languages of generated shifts given by the functional shifts whose languages belong 
to ^. Then ^ C 

Proof. Let L be a language in and X be a shift space defined by B{X) = L. By 
theorem 14. H there is a functional shift such that B{X) = B{J^) and Xjr = X. Then 
L G^. I 

From corollarv l4.21 any class of the languages of functional shifts is contained in that 
of generated shifts given by them. However, there is still the open problem of whether 
there exists a class of languages of functional shifts which is proper subset of languages 
of shift spaces generated by the functional shifts. We can study the relationship between 
the languages of functional shifts and those of generated shifts to bring this problem 
into focus. 

Hereafter, the next lemma is the key ingredient in each proof. 
Lemma 4.3 Let 

P^(n, x) = {y = ax\a G 3/ G i3|^|_i(.F) y^+i = fi{y,)}. (19) 
Then x G i3(X^) iff lim„,^oo I^A^^ x) ^ 0. 
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Proof. Suppose that x G B{Xjr), and y G Xjr having subblock x. There is a bi-infinite 
sequence f ^ T such that = fiiyi) for any z G Z. Since x is a subblock of y, an 
integer k such as x\ = yu exists. Hence yu-n ■ ■ ■ yu-iXi ■ ■ ■ x\x\ G T>jr{n, x) for any ri G N, 
so that hm„^oo T^rij^) ^ 0. 

Conversely, suppose that Yrnin^ooT^Tin^x) ^ 0. Then there is an infinite sequence 
y = ■ ■ ■ a_iaQXi ■ ■ ■ x\x\ (by Koenig's lemma) and a bi-infinite sequence / G ^ such that 
yi+i = fi{yi) for —oo < i < \x\. If a bi-infinite sequence z is defined by 



then z G Xjc- because Zi^i = fi{zi) for any i G Z. Since x is a subblock oi z, x E B{Xjr). 
I 

4.I. Shifts of finite type and sofic shifts 

Here we study the case in which a functional shift is a shift of finite type or a sofic shift. 
We first define shifts of finite type and sofic shifts. 

A shift of finite type is a shift space that can be described by a finite set of forbidden 
blocks. Although shifts of finite type are the simplest shifts, they are significant in the 
dynamical systems theory. If a dynamical system is hyperbolic, then the system has a 
Markov partition and a topological conjugacy to a shift of finite type. 

Sofic shifts are defined by using graphs, called labeled graphs, whose edges are 
assigned labels. A graph G consists of a finite set V = V{G) of vertices (or states) 
together with a finite set S = £{G) of edges. Each edge e G ^ starts at a vertex denoted 
by i{e) G V{G) and terminates at a vertex t{e) G V{G) (which can be the same as i{e)). 
Equivalently, the edge e has an initial state i{e) and a terminal state t(e). A labeled 
graph ^ is a pair (G, £), where G is a graph with edge set £, and the labeling C : £ ^ A 
assigns each edge e of G to a label C{e) in A. Let Xg be denoted by 



A subset X of the full shift A?" is a sofic shift if X = Xg for some labeled graph Q. Since 
a labeled graph is regarded as a state diagram of a finite state automaton, the language 
of a sofic shift is regular. 

It is known that a shift space is sofic iff it is a factor of a shift of finite type |18j . 
Since an identity function on a shift space is a factor code, shifts of finite type are sofic. 
Moreover, the class of sofic shifts is larger than that of shifts of finite type, because not 
all sofic shifts have finite type. For example, the even shift, which can be described by 
the collection {10^"'*"^1|?2 > 0} of forbidden blocks, is a sofic shift that does not have 
finite type. 

Let us prove the following theorems as the case in which functional shifts are shifts 
of finite type or sofic shifts. 

Theorem 4.4 (1) If X is a sofic shift, then there is a functional shift such that T 
has finite type and X = Xjr. 




if i < 
otherwise. 



(20) 



Xg = {xe A^\ 3ee£^WieZ t{ei) = i{ei+i) AXi = C{ei)}. 



(21) 
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(2) If a functional shift is sofic, then Xjr is sofic (so that if a functional shift J-" has 
finite type, Xjr is sofic). 

Proof. (1) Suppose that X is a sofic shift over A, and Q = {G, C) is a labeled graph 
such that X = Xg. If a is an edge of G, then fa:Au£U{5}^Au£U{5] (where 
^ n £: = and 5 ^ ^ U £:) is defined by 

^"'^ ^ ^ C{a) otherwise. ^ ^ 

Let F = {fa\ a G f } and F = {fafb G F^j t{a) ^ i{b)}. Recall that i{a) is an initial 
state and t{a) is a terminal state of a. If a functional shift JF over F can be described 
by F, then Xjr = X. Since F is a finite set, JF is a shift of finite type. 

(2) Suppose that is a sofic shift over F which is a set of maps on A, and F' is a 
collection of maps on F. By (1), there is a functional shift JF' over F' such that JF' has 
finite type and Xjr, = JF. Let 

X = {{x, f,g) e {Ax Fx F'f\ G r, W G Z = /,(x,) A = f/,(/,)}, (23) 

be a set of elements which are sequences of 3-tuples (■ ■ ■ (xq, /o, go){xi, fi,gi) ■ ■ ■), and 
F be a finite set of forbidden blocks such that X^ = T' . Then 

F = {{x, f,g)e{AxFx F'f\ ^ fo{xo) V A ^ go{fo)} 

U {{xJ,g)eiAxFxFr\geF} (24) 

is a set of forbidden blocks such that Xp. = X. Here, F is a finite set because F is finite. 
Suppose that ^ : {A x F x F') — > ^ is a 1-block map such that $((x, /, g)) = x. Since 
a sliding block code : X — * Xjr induced by $ is onto, is a factor code. If a shift 
space is a factor of a shift of finite type, then it is sofic. Thus Xjc- is a sofic shift. | 
The next is an instance satisfying theorem 14.41 (1). 

Example 4.5 Let A = {0, 1}, and F = {fa, fb, fc} be a set of functions on A such that 



X fa{x) fb{x) fc{x) 



10 
110 1 



If a functional shift JF can be described by a set of forbidden blocks F 
{fafc, fbfa, fbfb, fcfc}, then JF is a shift of finite type and X^ is the even shift. 



4-2. Context free languages 

This subsection studies the case in which the language of a functional shift is a context 
free language. The shift dynamics on some shift spaces with languages that are context 
free have long-range correlations, because stacks can hold memories infinitely. Moreover, 
if the shift spaces are probability measure spaces, the phase transition often appears in 
this dynamics jT^ . 

We begin by proving that if the language of a functional shift JF is context free, then 
there is a number p G N such that for any x & A* Vjr^p, x) 7^ iff lim„^oo T^rij^, x) 7^ 0, 
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by using the pumping lemmaf. Next we prove that if the language of is context free, 
then that of Xj^ is so. 

Lemma 4.6 Suppose that ^ is a functional shift and B{J^) is a context free language. 
There is a natural number p such that 

Vx e A* V^{p, x)^$^ lim Vr{n, x) ^ 0. (25) 

n—^oo 

Proof. Let G = {N, F, P, S} be a context free grammar such that = L[G), where 
L{G) denotes a formal language generated by G. Suppose, without loss of generality, 
that G is in Chomsky normal form|. Now a formal grammar G' = {N',A,P',S'} is 
defined as follows. A set A^' of nonterminal symbols is equal to {/lab|^ G A^, a, 6 e A}. 
P' is a set of productions determined by the following rules: 

• S" — > aSab G P' for any a.b E A; 

• Aab BacCcb e P' iff A ^ BC e P, where A,B,CeN and a,b,ce A; 

• Aab ^ b e P' iS A ^ f e P and /(a) = b, where A e N, f e F, and a,b e A. 
Clearly, G' is a context free grammar, furthermore, 

L{G')= U V^{0,x)^ U \JVr{n,x) (26) 

because x G L{G') iff there is a block / G L{G) = B{T) such that |/| = |a;| — 1 and 
Xi+i = fi{xi) for 1 < i < \x\. 

Prom the pumping lemma, there is a natural number p such that if r = uvwxy G 
L{G') and |r| > p then 

• \vx\ > 1, 

• \vwx\ < p, 

• for any i > 0, uv^wx^y G L{G'). 

Hence if rs G L{G'), i.e., V^{p, s) ^ 0, then Vj.{p + \vx\n,s) for all n G N. 
Note that if ©^(n, s) and m < n then X>^(m, s) ^ 0. Therefore, Vj^{p, s) iff 
lim„_^oo s) 7^ 0. I 

Since we use a nondeterministic pushdown automaton (NPDA) to prove the 
next theorem, we give the formal definition of NPDA. A NPDA is a 6-tuple M = 
{Q,T,,r,6,qo, Z, E}, where Q is a finite set of states, S is an alphabet (defining what 
set of input strings the automaton operates on), P is a stack alphabet (specifying the 
set of symbols that can be pushed onto the stack), 6 : Q x [HU {e}) x P — 2'^^'" is 

f In the theory of formal languages, the pumping lemma provides necessary conditions for languages 
to be context free. The pumping lemma for context free languages is as follows: if language L is context 
free, then there is a natural number p such that if r = uvwxy G L and \r\ > p then \vx\ > 1, \vwx\ < p, 
and for any i > 0, uv^wx^y £ L. 

I A formal grammar G = {Vn, Vt, P, S) is in Chomsky normal form iff all productions are of the form 
A BC or ^ — » a, where A,B,C G Vn and a € Vr- Every formal grammar in Chomsky normal is 
context free, and conversely, every context free grammar that does not generate an empty string can 
be transformed into an equivalent one which is in Chomsky normal form. 
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a transition function, go ^ Q is a starting state, Z G F is a starting stack symbol, and 
C Q is a set of final (or accepting) states. 
Given a NPDA M = {Q, E, T, 6, go, Z, E}, the relation he Q x S* x T* x Q x S* x T* 
is defined by 

(p, p) e 5{q, a, A) ^ (g, aw, Aj) h {p, w, ^7), (27) 
and the reflexive and transitive closure h is denoted by h. M accepts an input string x 

M M 

if there are p E E and 7 G F* such that {qo,x,Z) h (p, e, 7). The language recognized 
by M is the set 

L(M) = {xGS*| 3pEE3^ET* (go,x,Z) h (p,e,7)}. (28) 

It is known that a language L is a context free language iff L = L{M) for some NPDA 
M. 

Theorem 4.7 If JF is a functional shift and B{J^) is a context free language, then 
B{Xjr) is also a context free language. 

Proof. We will construct a NPDA which can recognize the language B{Xyr). 

Since S(.7-') is a context free language, B{J-')^ = {x^\x G 'B(JF)} is also context 
free, where x^ denotes the reversal of block x. Thus a NPDA M = {Q, F, F, 6, go, Z, E} 
to recognize B{J^)^ exists. Here a NPDA M' = {Q' , A,T' , 5' , Qq, Z' , E'} is defined as 
follows: let = {0, 1, ■ ■ ■ ,p} and 

• Q' = {Q X Ax N)U {q'o}, 

• F' = F U {Z'}, 

• E' = {{q,a,i) e Q'\q e E Ai = p}; 
next, 6' is determined by the foUowings: 

• ^'(go, a, Z') = {((go, a, 0), Z)} for any a G ^; 
. S'{q',,e,Z') = {{{qo,a,l),Z)\aeA}; 

• suppose that q E Q, a E A, and A G F; if 6 G ^ then 

5'((g, a, 0), 6, A) = {((r, 6, 0), (7)13/ G F (r, ^7) G 5(g, /, A) A /(6) = a}, (29) 
else if 6 = e, then 

5'((g,a,0),e,A) = {{{r, a,0), g)\{r, g) G (5(g,e,A)} 

U {((g,a,l),A)}; (30) 

• for any 1 < i < p, 

6\{q, a, ^), e, A) = {((r, 6, ^ + l),g)\b e A,3f e F (r, (?) G 6{q, /, A) A /(6) = a} 
U {((r,a,z),(7)|(r,(7)G5(g,e,A)}. (31) 
Given an input x G A*, M' accepts x iff there are y = xa E ^I^'I+p and / G F'^'"'"'''"^ 
such that M accepts / and yi^i = fiiyi) for 1 < i < \y\. Thus y^ G r'jc-(p, x^), so 
that M' accepts x iff X>jf(p, x^) 7^ 0. By lemma there is a natural number p such 
that ©^(p, x^) 7^ iff lim„_oo 'D^in, x^) ^ 0. Hence there is a NPDA M' such that M' 
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accepts X iff G B{Xjr) by lemma Accordingly, B{Xjr) is a context free language. | 



4-3. Context sensitive, recursive, and r.e. sets 

Let us consider the case in which the language of a functional shift J-" is not context 
free. A class of dynamical systems, called generalized shifts, has been proposed by 
Moore JU] , as corresponding to the class of languages more complicated than context- 
free languages. The class of generalized shifts is equivalent to that of Turing machines, 
and they can be embedded in smooth maps in or smooth flows in M^. 

In this case, the problem of determining the predicate x G B{Xjr) is more difficult 
than in the above subsection. This is because there does not necessarily exist a number 
p to satisfy that for any x E A* Vyr(q,x) 7^ iff lim^^oo x) 7^ in the case in 
which the language of a functional shift is not context free, while it always exists in 
context free cases. 

We suppose that F is a set of bijections and JF is a functional shift over F. 
From the definition of Pjf, it is clear that Djf(0,x) 7^ is a sufficient condition for 
lim^^oo T^j^i^i x) 7^ 0. Then we can prove the following theorems. 

Theorem 4.8 Let F be a set of bijections on A, and be a functional shift over F. 
If is a context sensitive language, then B{Xjr) is context sensitive. 

Proof. Let G = {N, F, P, S} be a context sensitive grammar to generate B{J^). 
G' = {N', A, P', S'} is determined by the following conditions: 

• N' = NUFU {S'}, where ^' ^ U F. 

• P' contains only productions satisfying the following restrictions: 

— for any a & A, S' aS & P' and S' a E P'; 

— for any a,l3 e{NU F)*, if a ^ /3 G P then a ^ [3 e P'; 

— if / G F and a & A, then af G P'; 

Then G' is a context sensitive grammar because if a — > /5 G P' then |a| < \f3\. 

Let X G L{G'). Since there is a block / G L{G) = B{J-') such that |/| = |x| — 1 
and Xj+i = fi{xi) for 1 < i < |a;|, then Vjr(0,x) 7^ 0. Thus lim„^oo x) 7^ 
because every function in F is a bijection, so that x G B{Xjr) by lemma 14.31 Hence 
L{G') C B{X^). 

Conversely, let x G B{Xjr). Clearly there is a block / G i3|a;|_i(jF) such that 
Xj+i = fi{xi) for any 1 < z < |x|. Thus / can be derived from S in G. Then 

S' =^ xiS 

G' 



G' 



xif 

^ X, (32) 

G' 

SO that X G L{G'). Hence B{X^) C L{G'). Accordingly L{G') = B{X^) and B{X^) is 
a context sensitive language. I 
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Theorem 4.9 Let F be a set of bijections on A, and JF be a functional shift over F. 
If B{J^) is r.e., then B{X^) is r.e. 

Proof. Suppose that G is a type-0 grammar to generate B{J-'), and G' is defined in 
the similar way to the proof of theorem 14.81 As discussed in that proof, G' is also a 
type-0 grammar and L{G') = B{Xjr). i 

In the general case in which some functions in F may be not bijections, it is 
difficult to determine where the languages of generated shifts are located in the Chomsky 
hierarchy. To locate a collection of forbidden blocks is, however, an easier task than 
studying this problem. 

Theorem 4.10 Suppose that JF is a functional shift and is context sensitive. 

Then there is a context sensitive language F of forbidden blocks such that Xy = Xj:. 

Proof. Let 

F = {x e A*\V:f{0,x) = ^}. (33) 

Now we show that F is a collection of forbidden blocks such that Xp = Xjr. Suppose 
that X is a bi-infinite sequence in such as x ^ Xjr. Then there is a block y, which is 
not contained in B{Xjr), occurring in x . Thus, by lemma a natural number n such 
as Vjriji^ = exists. For any a G A^, ay E F because I'jc-(0, ay) = 0. Since there is a 
block a G A"' such that x contains ay, some blocks in F occur in x. Conversely, suppose 
that X G A^ contains a block y G F. Then it is clear that x ^ Xjr. Accordingly, x ^ Xjr 
iff there is a block in F which occurs in x. Hence Xp = Xjr. 

Next, we will explain the existence of a linear bounded automaton (LB A) which 
can recognize F. Let Mi be a LBA to compute the following function 

/X f 1 if a; G B{J^), ,,,, 

^^^) = |o .ix^Bln ^''^ 

We construct a LBA M2 with two separate tapes as follows. First, a block x G ^* is 
inscribed on tape-1, and / G F^^^~^ on tape-2 (see Figure P). Then M2 carries out the 
following operations. 

(1) The machine M2 begins with the head resting in anticipation on the left most cell. 
The machine repeatedly moves right and reads the cell value beneath the head until 
the right most cell. When the machine finds i such that Xj+i 7^ fi{xi), M2 accepts 
{x, f) and halts. 

(2) The machine M2 calls the subroutine Mi with the block / on the tape-2, which 
returns the answer "0" or "1" as appropriate. If the answer is "0", that is. Mi does 
not accept /, then M2 accepts {x, f). In the other case, M2 does not accept (x, /). 

Now we consider a machine M such that, for any input x, if M2 accepts (x, /) for all 
/ G Fl^l-i then M accepts X. Since to construct a LBA to enumerate F'^' ^ is an 
easy task, from this explicit definition we can get the LBA M. For any x, M accepts 
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X iff Vjr(0,x) = because there is no / G B^^i-ii^) such that Xj+i = fi{xi) for any 
1 < i < \x\. Hence M is a LBA to recognize F. I 
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Figure 1. Illustration of the linear bounded automaton M2 



Moreover, by using a proof similar to theorem l4.1Ul we can prove that if the language 
of a functional shift is a recursive set, then F is also recursive. A language is a recursive 
set if there exists a Turing machine which recognizes the language and always halts. 

Theorem 4.11 Suppose that is a functional shift and B{J-') is recursive. Then there 
is a recursive set F of forbidden blocks such that Xp = Xjr. 

Proof. Since B{T) is a recursive set, there is a Turing machine M3 to compute 
the function (p in equation (jH!)). Suppose that M2 in the proof of theorem 14.101 calls 
subroutine M3 instead of Mi. Then M2 and M are Turing machines that always halt 
after a finite amount of time, and M recognizes F. Thus, F is a recursive set. 1 

4.4- The language of a generated shift beyond r.e. 

In this subsection, we prove that there is a functional shift JF satisfying that B{J^) is r.e. 
and B{X:p) is not r.e. Here a set A is r.e. iff the predicate x G A is partially decidable, 
i.e., the partial characteristic function 



(35) 



1 if X G A, 

undefined ii x ^ A, 

is computable. Note that if a predicate P(x) is partially decidable and undecidable, 
then -iP(x) is not partially decidable. For example, the following predicate 

'a Turing machine of the Godel number x eventually stops on input x' 

is partially decidable and undecidable. In the proof of the next theorem, we show that 
X G B{Xjr) iff a Turing machine of the Godel number x never halts on input in order 
to prove X G B{Xjr) is not partially decidable. 

Theorem 4.12 There is a function shift JF such that B{J^) is r.e. and B{Xjr) is not r.e. 

Proof. For any x G {01"5|n G N}, let 

num{x) = the number of I's occurring in x (36) 
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and a Turing machine of the Godel number n be denoted by T„. We will show a 
functional shift JF such that i3(jF) is r.e. and 

X e B{Xjr) Tnum{x) never halts on input x. (37) 

Let A = {0, 1, 6}, F = {/, g, h} be a set of maps defined by 
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and JF be a functional shift over F such that is r.e. Suppose that a Turing machine 
M to recognize satisfies the following conditions: 

• Tnum(x) does uot halt on input x before time t iff M accepts f*g'^^'^i^)f^ 

• Tnum(x) halts on input x at time t iff M accepts }iftgnum{x) j ^ 

where x G {Ol^^lra G N}. Since it is clear that num{x) and the emulation of Tnum{x) are 
in fact computable functions, M exists, by Church's thesis. 

For any x G {Ol^^lri G N}, Vjr(t,x) 7^ iff Tnum,(x) does not halt on input x before 
time t + 1. Accordingly, 

Tnum{x) never halts on input x 4^ lim Vjr{t, x) 7^ 

t — ^00 

^xGi3(X^), (38) 

by lemma Hence the predicate x G B{Xjr) is not partially decidable, so that B{Xjr) 
is not r.e. I 

5. Discussion 

To study the relationship between dynamics and meta-dynamics, we have compared 
functional shifts with generated shifts, in both dynamical systems and computational 
terms. In section El we have proved that the entropy of a functional shift gives the 
upper limit for that of a generated shift given by the functional shift. In other words, 
every functional shift is at least as complex as the generated shift from the standpoint of 
dynamics. In section I5 considering the language of a shift space as a formal language, 
we have shown that there is a case in which the language of a functional shift is simpler 
than that of a shift space generated by the functional shift. Figure |2l shows the summary 
of results, in which the languages of functional shifts and generated shifts are located 
in the Chomsky hierarchy. It shows clearly that the class of the languages of functional 
shifts is the same as or smaller than that of the generated shifts given by them. From 
those results, we may consider that the complexity of dynamics does not correspond 
to that of computation under the generative operation introduced here. Moreover, the 
viewpoints of both dynamics and computation give us opposite results concerning the 
complexity of systems. This means that an analysis of the complexity of systems surely 
depends on how we select a measure of complexity. Thus it is important to study 
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dynamical systems from several viewpoints, for example, those of both dynamics and 
computation, when we analyze the complexity of the systems. 



partially decidable 
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^NPDA/CFG^ 
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NSPACE (n) 
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Figure 2. The computational hierarchy of the languages of functional shifts and 
generated shifts. Every class of the languages of functional shifts is the same as or 
smaller than that of generated shifts. For functional shifts with r.e. language, we have 
generated shifts whose language is beyond r.e. The solid arrow from A to J5 denotes 
that A is a class of the languages of functional shifts and B is that of shift spaces 
generated by the functional shifts. The broken arrow from A to B denotes that A 
is the same as the case of solid arrow and i? is a class of sets of forbidden blocks 
which can describe generated shifts given by the functional shifts. D = deterministic, 
N — nondeterministic, U — universal, G — grammar, A — automata, TM = Turing 
machine, RE = recursively enumerable, LBA = linear bounded A, PDA = pushdown 
A, FA = Finite A, CSG = context sensitive G, CFG = context free G, RG = regular 
G. 



It is interesting that the class of languages of functional shifts is equal to that of 
generated shifts, in the case in which the languages of the functional shifts are regular 
or context free, while there is no equivalence in the r.e. cases. The cause is conjectured 
to be that the equivalency depends on whether there exists a natural number n such 
that T>jr{n,x) 7^ iff x G B{Xjr) for any x G A*. Lemma (4.61 and theorem 14.71 confirm 
our presumption. Theorem 14. 81 and theorem 14. 91 also support it, because if any functions 
are bijections then Vj:{Q,x) 7^ iff s G B{Xjr). 

Let us discuss the existence of systems in which some predicates of dynamical 
systems theory are not partially decidable. In theorem I4.12| we have proved that there 
is a functional shift T satisfying that the language of T is r.e. and that of Xjr is not r.e. 
Note that the predicate x G B{X) means that the subset {y G y[i,\xW = x} of 
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contains part of an invariant set X. Many problems about dynamical systems can be 
resolved into this predicate. For example, we can consider such problems as follows: 

(1) Given open sets Y, Z G X , is there a point z & Y that falls into Z under the shift 
map on X? 

(2) Is the shift map on X topologically transitive? 

Problem (1) is considered to be a prediction problem of orbits in a dynamics system. The 
reason why problem (1) resolves itself into the predicate x G S{X) is that if m, f G B{X), 
Y = {y E X\ = u} and Z = {y E X\ y[i^\v\] = v}, then there is a block w such 

that uwv G B{X) iff (1) is true. Since problem (2) can be reduced to (1), problem 
(2) contains the predicate as a subproblem. Thus, a shift space Xjc- is so complex that 
those predicates are not necessarily partially decidable, even if such predicates of are 
partially decidable. As a dynamical system in which those problems are not partially 
decidable, we may consider the system with riddled basins [201 121] ■ In fact, probably 
no algorithm exists which is able to assess problem (1) in a finite number of steps if Z 
is a riddled basin jTHl 121] • 

Generally, the automaton to recognize a set not to be r.e. is called a super- Turing 
machine [221 12n]- Every super- Turing machine recognizes a set to be beyond r.e. by 
using infiniteness, for example the property of the real number, which usual Turing 
machines do not have. Since the languages of shift spaces generated by functional 
shifts with r.e. languages are not r.e., those shift spaces have relevance to super- Turing 
machines. Some classes of sets to be beyond r.e. have been discussed in the field of analog 
computation [TTJ [T21 dSl HH 113 121] ■ Hamkins and Lewis, for instance, analyze classes of 
computations with infinitely many steps, and investigate computability and decidability 
on the reals [21]. Notice that shift spaces are usually continuums, because every shift 
space is defined as a collection of bi-infinite sequences. Hence, constructing a generated 
shift from a functional shift is an operation on a continuum, including tasks such as 
infinite mapping Xj+i = fi{xi) for all z G Z. This operation, which is implicitly involved 
in the definition of functional shifts and generated shifts, often affects properties of the 
shifts themselves. For example, by using such an operation, we show that the predicate 
X G B{Xjr) is not partially decidable in the proof of theorem 14.121 Therefore, our 
framework could be related to super- Turing machines and analog computation. Further 
analysis from the viewpoints of them is a future research topic. 

We have introduced the framework of functional shifts as a model of dynamic change 
of functions. Since we can consider that a bi-infinite sequence of function {fi)i^z denotes 
the evolution of maps, functional shifts represent dynamics of functions. Generated 
shifts also represent the dynamics determined by Xj+i = fi{xi). Considering the shift 
map on Xjr as dynamics, we can regard that on JF as meta-dynamics. Let us focus on 
the operation to generate shift spaces from functional shifts, i.e., to construct dynamics 
from meta-dynamics. By theorem 14.121 such operation includes a task to generate sets 
not to be r.e. in spite of the fact that any functions in F are computable. Thus, as we 
discussed, when we study dynamical systems modified by meta-dynamics, the existence 
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of complex dynamics should be taken into account. 

Finally, we consider self-modifying systems, in which rules governing a system 
are used to change the rules themselves. We have difficulty in achieving a direct 
representation of the dynamics of a self-modifying type. The cause of the difficulty 
is that functions and states cannot be perfectly separated in self-modifying systems. 
Research into the function dynamics of the self-modifying type has recently become a 
subject of special interest in the study of these systems. Functional dynamics is an 
example of a self-modifying system, because the change of function / is determined by 
a self-reference term fof As another example, objects in algorithmic chemistry 

encode functions which change the objects themselves [3 IE]- To describe self-modifying 
systems, we must take the self- referential nature of a dynamical system into account. We 
can represent this characteristic using functional shifts as follows [2^]. Let us consider 
that a conjugacy from a functional shift to a generated shift given by the functional shift 
is a 'self- reference code' between functions and states. Considering that each sequence 
of functions is equal to a sequence of symbols corresponding to itself under the code, we 
can regard the functional shift as self-modifying. For instance, the functional shift 
given in example 12. HI or ITHl has conjugacy to Xjc, so that the functional shifts in these 
examples are regarded as self-modifying systems. However, the relationship between the 
systems described by functional shifts and other systems introduced by jH 13 E] is 
not clear. 

6. Conclusion 

We have investigated shift dynamics called functional shifts within both dynamics and 
computational frameworks. From the dynamical viewpoint, we have proved that the 
entropy of a functional shift is not less than that of a shift space generated by the 
functional shift. This means that functional shifts generate less complex shift spaces 
than themselves. On the other hand, we have compared functional shifts with generated 
shifts in terms of the Chomsky hierarchy (see Figure EI). We have proved that any class 
of the languages of shift spaces is at least as large as that of the functional shifts that 
generate the shift spaces. Furthermore, we have shown that there is a class of the 
languages of functional shifts, which is strictly smaller than that of generated shifts 
given by the functional shifts. From those results, we have argued that the viewpoints 
of dynamics and computation give us opposite results about complexity of systems. 

We have shown a new class of shift spaces, generated shifts whose languages are 
not r.e. if the languages of functional shifts to give the generated shifts are r.e. The shift 
map over some shift spaces in the class has very unpredictable dynamics. This new class 
gives us a way to study dynamical systems from the viewpoint of analog computation. 
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Appendix A. Review of the Chomsky hierarchy 

The Chomsky hierarchy is a containment hierarchy of classes of formal grammars that 
generate formal languages. This hierarchy was described by Noam Chomsky |27j. 
A formal grammar (or type-0 grammar) is a 4-tuple (Vat, Vt, P, S), where 

• Vn is a finite set of nonterminal symbols; 

• Vt is a finite set of terminal symbols such as Vat H Vj- = 0, and = Vat U is 
called the set of grammar symbols; 

• P is a finite collection of productions which are of the form a ^ f3 with a G 
and P G V*, 

• G Vat is a designated symbol called the start symbol. 

Given a formal grammar G = {Vn,Vt, P, S), the derivation relation =^G V* x V* is 

G 

defined by 

7^5 ^7/55 iff a^(3eP, (A.l) 

G 

where a, /3, 7, 5 G V*. The transitive closure of =^ is denoted by and the reflexive 
and transitive closure of =^ is denoted by The language generated by G is the set 

L{G) = {w eV^\S ^w}. (A.2) 

G 

A language L C is a formal language (or type-0 language) iff L = L{G) for some 
formal grammar G. 

The Chomsky hierarchy consists of classes of regular grammars, context free 
grammars, context sensitive grammars, and type-0 grammars. Regular, context free, 
and context sensitive grammars are more restrictive than formal grammars. 

• A regular grammar (type-3 grammar) is a formal grammar G = (Vat, Vt, P, S), such 
that the productions are of the form a ^ P with a E Vn and /5 G Vt U Vr x Vat. A 
language L C V^ is a regular language iff L = L{G) for some regular grammar G. 

• A context free grammar (type-2 grammar) is a formal grammar G = (Vat, Vt, P, S), 
such that the productions are of the form a — > /5 with a G Vat and /5 G V~^. A 
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language L C V-^ is a, context free language iff L — L{G) for some context free 
grammar G. 

• A context sensitive grammar (type-1 grammar) is a formal grammar G = 
{Vn, Vt, P, S), such that the productions are of the form a — > /3 satisfying \a\ < \f3\. 
A language L C is a context sensitive language iff L = L{G) for some context 
sensitive grammar G. 

From the above definition, every regular grammar is context free, every context free 
grammar is context sensitive and every context sensitive grammar is type-0. Moreover, 
these are all proper inclusions. 

It is known that there exists automaton corresponding to each formal language 
belonging to the Chomsky hierarchy. Every type-0 language can be recognized by a 
Turing machine, where a Turing machine is a finite state machine moving left and 
right on a tape, on which a string of symbols in some finite alphabet is written. The 
language recognized by a Turing machine is defined as all the strings on which it halts. 
These languages are also known as the recursively enumerable (r.e.) languages. Every 
context sensitive language can be recognized by a linear bounded automaton which is 
a nondctcrministic Turing machine whose tape is bounded by the length of the input 
string. Every context free language can be recognized by a nondeterministic pushdown 
automaton which is a finite state automaton having a stack. Every regular language can 
be recognized by a finite state automaton which has no memory. Thus, the complexity 
from the Chomsky hierarchy is based on the memory size of automata to recognize 
languages. The table 1 summarizes each of the Chomsky's four types of grammars, the 
class of languages each grammar generates, and the type of automaton that recognizes 
each language. 

Table 1 Summarize each of the Chomsky's four types of grammars, languages, and 



automata. 



Grammar 


Language 


Automaton 


Type-0 


Recursively enumerable 


Turing machine 


Type-1 


Context sensitive 


Linear bounded automaton 


Type-2 


Context free 


Nondeterministic pushdown automaton 


Type-3 


Regular 


Finite state automaton 



